Less Is More: Robust Robot Learning via Partially Observable Multi-Agent Reinforcement Learning

In many multi-agent and high-dimensional robotic tasks, the controller can be designed in either a centralized or decentralized way. Correspondingly, it is possible to use either single-agent reinforcement learning (SARL) or multi-agent reinforcement learning (MARL) methods to learn such controllers. However, the relationship between these two paradigms remains under-studied in the literature. This work explores research questions in terms of robustness and performance of SARL and MARL approaches to the same task, in order to gain insight into the most suitable methods. We start by analytically showing the equivalence between these two paradigms under the full-state observation assumption. Then, we identify a broad subclass of \textit{Dec-POMDP} tasks where the agents are weakly or partially interacting. In these tasks, we show that partial observations of each agent are sufficient for near-optimal decision-making. Furthermore, we propose to exploit such partially observable MARL to improve the robustness of robots when joint or agent failures occur. Our experiments on both simulated multi-agent tasks and a real robot task with a mobile manipulator validate the presented insights and the effectiveness of the proposed robust robot learning method via partially observable MARL.Comment: 8 pages, 8 figure

MSS U-Net: 3D segmentation of kidneys and tumors from CT images with a multi-scale supervised U-Net

Accurate segmentation of kidneys and kidney tumors is an essential step for radiomic analysis as well as developing advanced surgical planning techniques. In clinical analysis, the segmentation is currently performed by clinicians from the visual inspection of images gathered through a computed tomography (CT) scan. This process is laborious and its success significantly depends on previous experience. We present a multi-scale supervised 3D U-Net, MSS U-Net to segment kidneys and kidney tumors from CT images. Our architecture combines deep supervision with exponential logarithmic loss to increase the 3D U-Net training efficiency. Furthermore, we introduce a connected-component based post processing method to enhance the performance of the overall process. This architecture shows superior performance compared to state-of-the-art works, with the Dice coefficient of kidney and tumor up to 0.969 and 0.805 respectively. We tested MSS U-Net in the KiTS19 challenge with its corresponding dataset.</p

HORMETIC EFFECTS OF ACUTE METHYLMERCURY EXPOSURE ON GRP78 EXPRESSION IN RAT BRAIN CORTEX

This study aims to explore the expression of GRP78, a marker of endoplasmic reticulum (ER) stress, in the cortex of rat brains acutely exposed to methylmercury (MeHg). Thirty Sprague-Dawley (SD) rats were randomly divided into six groups, and decapitated 6 hours (h) after intraperitoneal (i.p.) injection of MeHg (2, 4, 6, 8 or 10 mg/kg body weight) or normal saline. Protein and mRNA expression of Grp78 were detected by western blotting and real-time PCR, respectively. The results showed that a gradual increase in GRP78 protein expression was observed in the cortex of rats acutely exposed to MeHg (2, 4 or 6 mg/kg). Protein levels peaked in the 6 mg/kg group (p \u3c 0.05 vs. controls), decreased in the 8 mg/kg group, and bottomed below the control level in the 10 mg/kg group. Parallel changes were noted for Grp78 mRNA expression. It may be implied that acute exposure to MeHg induced hormetic dose-dependent changes in Grp78 mRNA and protein expression, suggesting that activation of ER stress is involved in MeHg-induced neurotoxicity. Low level MeHg exposure may induce GRP78 protein expression to stimulate endogenous cytoprotective mechanisms